<!DOCTYPE html>
<html>

<head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
	<meta name="theme-color" content="#33474d">
	<title>LVS原理详解及部署之五：LVS+keepalived实现负载均衡&amp;高可用 | 失落的乐章</title>
	<link rel="stylesheet" href="/css/style.css" />
	
      <link rel="alternate" href="/atom.xml" title="失落的乐章" type="application/atom+xml">
    
</head>

<body>

	<header class="header">
		<nav class="header__nav">
			
				<a href="/archives" class="header__link">Archive</a>
			
				<a href="/tags" class="header__link">Tags</a>
			
				<a href="/atom.xml" class="header__link">RSS</a>
			
		</nav>
		<h1 class="header__title"><a href="/">失落的乐章</a></h1>
		<h2 class="header__subtitle">技术面前，永远都是学生。</h2>
	</header>

	<main>
		<article>
	
		<h1>LVS原理详解及部署之五：LVS+keepalived实现负载均衡&amp;高可用</h1>
	
	<div class="article__infos">
		<span class="article__date">2017-10-12</span><br />
		
		
			<span class="article__tags">
			  	<a class="article__tag-link" href="/tags/LVS/">LVS</a>
			</span>
		
	</div>

	

	
		<h2 id="一、实验环境需求-amp-准备"><a href="#一、实验环境需求-amp-准备" class="headerlink" title="一、实验环境需求&amp;准备"></a>一、实验环境需求&amp;准备</h2><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;这次实验要完成的一个架构如下图所示，通过LVS-DR-MASTER,LVS-DR-BACKUP作为LVS负载均衡调度器，并且两者之间通过keepalived来两者之间的HA。keepalived本身就是为了LVS为开发的，所以说通过keepalived来进行LVS的配置就显得十分的方便。而且keepalived是直接操作ip_vs不用通过ipvsadm，所以更加方便。</p>
<ol>
<li>实验架构图&amp;需求表：</li>
</ol>
<p><figure class="figure"><img src="https://github.com/hcldirgit/image/blob/master/LVS%E5%8E%9F%E7%90%86%E8%AF%A6%E8%A7%A3%E5%8F%8A%E9%83%A8%E7%BD%B2%E4%B9%8B%E4%BA%94%EF%BC%9ALVS+keepalived%E5%AE%9E%E7%8E%B0%E8%B4%9F%E8%BD%BD%E5%9D%87%E8%A1%A1&amp;%E9%AB%98%E5%8F%AF%E7%94%A8/01.jpeg?raw=true" alt=""></figure></p>
<table>
<thead>
<tr>
<th>角色</th>
<th>IP地址</th>
<th>备注</th>
</tr>
</thead>
<tbody>
<tr>
<td>主LVS调度器（MASTER)</td>
<td>192.168.41.181</td>
<td>使用keepalived配置备</td>
</tr>
<tr>
<td>LVS调度器（BACKUP)</td>
<td>192.168.41.25</td>
<td>&#160;</td>
</tr>
<tr>
<td>1HTTP服务器（RS1)</td>
<td>192.168.41.31</td>
<td>apache服务器（一般生产环境需要外网IP地址，这里用内网IP地址替代）</td>
</tr>
<tr>
<td>HTTP服务器（RS2)</td>
<td>192.168.41.33</td>
<td>&#160;</td>
</tr>
<tr>
<td>虚拟IP地址（VIP)</td>
<td>192.168.41.249</td>
<td>虚拟IP地址</td>
</tr>
</tbody>
</table>
<ol>
<li>部署http服务器，验证能正常访问</li>
</ol>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;这里就是要保证http能正常访问。</p>
<h2 id="二、配置keepalived实现负载均衡-amp-高可用"><a href="#二、配置keepalived实现负载均衡-amp-高可用" class="headerlink" title="二、配置keepalived实现负载均衡&amp;高可用"></a>二、配置keepalived实现负载均衡&amp;高可用</h2><h3 id="安装keepalived软件"><a href="#安装keepalived软件" class="headerlink" title="安装keepalived软件"></a>安装keepalived软件</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">wget http://www.keepalived.org/software/keepalived-1.2.8.tar.gz</div><div class="line">tar -zxf keepalived-1.2.8.tar.gz</div><div class="line"><span class="built_in">cd</span> keepalived-1.2.8</div><div class="line">./configure --prefix=/usr/<span class="built_in">local</span>/keepalived</div><div class="line">make</div><div class="line">make install</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;配置keepalived的自启动&amp;拷贝keepalived的执行程序</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">cp /usr/<span class="built_in">local</span>/keepalived/sbin/keepalived /usr/sbin/</div><div class="line">cp /usr/<span class="built_in">local</span>/keepalived/sbin/keepalived /usr/sbin/</div><div class="line">cp /usr/<span class="built_in">local</span>/keepalived/etc/sysconfig/keepalived /etc/sysconfig/</div><div class="line">cp /usr/<span class="built_in">local</span>/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;开启内核的转发功能</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">vi /etc/sysctl  net.ipv4.ip_forword = 1</div></pre></td></tr></table></figure>
<h3 id="配置LVS-DR-MASK的keepalived-conf配置文件"><a href="#配置LVS-DR-MASK的keepalived-conf配置文件" class="headerlink" title="配置LVS-DR-MASK的keepalived.conf配置文件"></a>配置LVS-DR-MASK的keepalived.conf配置文件</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div></pre></td><td class="code"><pre><div class="line">! Configuration File <span class="keyword">for</span> keepalived</div><div class="line">global_defs &#123;</div><div class="line"> notification_email &#123;</div><div class="line"> 752119102@qq.com <span class="comment">#设置报警邮箱，一般不再这做，而是用其他方式报警。</span></div><div class="line"> &#125;</div><div class="line"> notification_email_from keepalived@localhost <span class="comment">#设定发送邮件地址</span></div><div class="line"> smtp_server 127.0.0.1 <span class="comment">#设定发送邮件服务器</span></div><div class="line"> smtp_connect_timeout 30 <span class="comment">#设定SMTP连接超时时间</span></div><div class="line"> router_id LVS_DEVEL <span class="comment">#查阅说明文档得知route_id配置是为了标识当前节点，我将其设置为NodeA。当然两个节点的此项设置可相同，也可不相同。</span></div><div class="line">&#125;</div><div class="line">vrrp_instance VI_1 &#123; <span class="comment">#定义虚拟路由实例，不同实例ID不同。</span></div><div class="line"> state MASTER  <span class="comment">#定义服务器在keepalived中的角色主服务器</span></div><div class="line"> interface eth0 <span class="comment">#定义进行检测的端口eth0</span></div><div class="line"> virtual_router_id 51 <span class="comment">#定义虚拟路由ID，同一个实例的主从一样。</span></div><div class="line"> priority 100  <span class="comment">#定义在虚拟路由器组的权限，越大越高</span></div><div class="line"> advert_int 1 <span class="comment">#定义检测时间间隔</span></div><div class="line"> authentication &#123; <span class="comment">#定义认证方式密码，主从必须一样</span></div><div class="line"> auth_type PASS</div><div class="line"> auth_pass 1111</div><div class="line"> &#125;</div><div class="line"> virtual_ipaddress &#123; <span class="comment">#指定虚拟IP地址</span></div><div class="line"> 192.168.41.249</div><div class="line"> &#125;</div><div class="line">&#125;</div><div class="line">virtual_server 192.168.41.249 80 &#123; <span class="comment">#定义虚拟服务，需指定IP地址和端口，空格隔开。</span></div><div class="line"> delay_loop 6  <span class="comment">#定义RS运行情况监测时间间隔</span></div><div class="line"> lb_algo rr  <span class="comment">#定义负载调度算法</span></div><div class="line"> lb_kind DR <span class="comment">#定义LVS的工作模式</span></div><div class="line"> nat_mask 255.255.255.0 <span class="comment">#定义虚拟服务的mask</span></div><div class="line"> persistence_timeout 50 <span class="comment">#定义会话保持时间，S为单位</span></div><div class="line"> protocol TCP <span class="comment">#指定转发协议</span></div><div class="line"> real_server 192.168.41.31 80 &#123; <span class="comment">#定义真实服务器IP地址和端口</span></div><div class="line"> weight 1 <span class="comment">#定义RS的权重</span></div><div class="line"> TCP_CHECK&#123; <span class="comment">#RS server健康检查部分</span></div><div class="line"> connect_timeout 10 <span class="comment">#定义超出10s连接超时</span></div><div class="line"> nb_get_retry 3 <span class="comment">#定义重试次数</span></div><div class="line"> delay_before_retry 3 <span class="comment">#定义重试时间间隔</span></div><div class="line"> connect_port 80 <span class="comment">#定义健康检查端口</span></div><div class="line"> &#125;</div><div class="line"> real_server 192.168.41.33 80 &#123; </div><div class="line"> weight 1</div><div class="line"> TCP_CHECK&#123; </div><div class="line"> connect_timeout 10 </div><div class="line"> nb_get_retry 3 </div><div class="line"> delay_before_retry 3 </div><div class="line"> connect_port 80 </div><div class="line"> &#125;</div><div class="line">&#125;</div></pre></td></tr></table></figure>
<h3 id="配置LVS-DR-BACKUP的keepalived-conf配置文件"><a href="#配置LVS-DR-BACKUP的keepalived-conf配置文件" class="headerlink" title="配置LVS-DR-BACKUP的keepalived.conf配置文件"></a>配置LVS-DR-BACKUP的keepalived.conf配置文件</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div></pre></td><td class="code"><pre><div class="line">! Configuration File <span class="keyword">for</span> keepalived</div><div class="line">global_defs &#123;</div><div class="line"> notification_email &#123;</div><div class="line"> 752119102@qq.com <span class="comment">#设置报警邮箱，一般不再这做，而是用其他方式报警。</span></div><div class="line"> &#125;</div><div class="line"> notification_email_from keepalived@localhost <span class="comment">#设定发送邮件地址</span></div><div class="line"> smtp_server 127.0.0.1 <span class="comment">#设定发送邮件服务器</span></div><div class="line"> smtp_connect_timeout 30 <span class="comment">#设定SMTP连接超时时间</span></div><div class="line"> router_id LVS_DEVEL  <span class="comment">#负载均衡器标示，在局域网内是唯一的</span></div><div class="line">&#125;</div><div class="line">vrrp_instance VI_1 &#123; <span class="comment">#定义虚拟路由实例，不同实例ID不同。</span></div><div class="line"> state BACKUP  <span class="comment">#定义服务器在keepalived中的角色</span></div><div class="line"> interface eth0 <span class="comment">#定义进行检测的端口eth0</span></div><div class="line"> virtual_router_id 51 <span class="comment">#定义虚拟路由ID，同一个实例的主从一样。</span></div><div class="line"> priority 50  <span class="comment">#定义在虚拟路由器组的权限，越大越高</span></div><div class="line"> advert_int 1 <span class="comment">#定义检测时间间隔</span></div><div class="line"> authentication &#123; <span class="comment">#定义认证方式密码，主从必须一样</span></div><div class="line"> auth_type PASS</div><div class="line"> auth_pass 1111</div><div class="line"> &#125;</div><div class="line"> virtual_ipaddress &#123; <span class="comment">#指定虚拟IP地址</span></div><div class="line"> 192.168.41.249</div><div class="line"> &#125;</div><div class="line">&#125;</div><div class="line">virtual_server 192.168.41.249 80 &#123; <span class="comment">#定义虚拟服务，需指定IP地址和端口，空格隔开。</span></div><div class="line"> delay_loop 6 <span class="comment">#定义RS运行情况监测时间间隔</span></div><div class="line"> lb_algo rr <span class="comment">#定义负载调度算法</span></div><div class="line"> lb_kind DR <span class="comment">#定义LVS的工作模式</span></div><div class="line"> nat_mask 255.255.255.0 <span class="comment">#定义虚拟服务的mask</span></div><div class="line"> persistence_timeout 50 <span class="comment">#定义会话保持时间，S为单位</span></div><div class="line"> protocol TCP <span class="comment">#指定转发协议</span></div><div class="line"> real_server 192.168.41.31 80 &#123; <span class="comment">#定义真实服务器IP地址和端口</span></div><div class="line"> weight 1 <span class="comment">#定义RS的权重</span></div><div class="line"> TCP_CHECK&#123; <span class="comment">#RS server健康检查部分</span></div><div class="line"> connect_timeout 10 <span class="comment">#定义超出10s连接超时</span></div><div class="line"> nb_get_retry 3 <span class="comment">#定义重试次数</span></div><div class="line"> delay_before_retry 3 <span class="comment">#定义重试时间间隔</span></div><div class="line"> connect_port 80 <span class="comment">#定义健康检查端口</span></div><div class="line"> &#125;</div><div class="line"> real_server 192.168.41.33 80 &#123; </div><div class="line"> weight 1</div><div class="line"> TCP_CHECK&#123; </div><div class="line"> connect_timeout 10 </div><div class="line"> nb_get_retry 3 </div><div class="line"> delay_before_retry 3 </div><div class="line"> connect_port 80 </div><div class="line"> &#125;</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;说明：这里主LVS-DR-MASTER和LVS-DR-BACKUP之间的配置的差别就只有红色部分：HA的角色（MASTER,BACKUP)和优先级不同，还有router_id。</p>
<h3 id="客户端配置LVS参数"><a href="#客户端配置LVS参数" class="headerlink" title="客户端配置LVS参数"></a>客户端配置LVS参数</h3><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;客户端需要做的工作就是绑定我们的VIP在lo口，并且进行ARP抑制，之前的文章已经提过此方法咯。现在我们就换成将配置写成脚本来执行。</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;脚本内容：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div></pre></td><td class="code"><pre><div class="line">[root@RS2 ~]<span class="comment"># cat lvs-client.sh</span></div><div class="line"><span class="comment">#!/bin/bask</span></div><div class="line"><span class="comment"># 752119102@qq.com</span></div><div class="line"><span class="comment">#</span></div><div class="line">. /etc/rc.d/init.d/<span class="built_in">functions</span></div><div class="line">VIP=(</div><div class="line">192.168.41.249</div><div class="line">)</div><div class="line"><span class="keyword">function</span> <span class="function"><span class="title">start</span></span>()&#123;</div><div class="line"> <span class="keyword">for</span> ((i=0;i&lt;`<span class="built_in">echo</span> <span class="variable">$&#123;#VIP[*]&#125;</span>`;i++))</div><div class="line"> <span class="keyword">do</span></div><div class="line"> <span class="built_in">echo</span> <span class="variable">$&#123;i&#125;</span> <span class="variable">$&#123;VIP[$i]&#125;</span></div><div class="line"> ifconfig lo:<span class="variable">$&#123;i&#125;</span> <span class="variable">$&#123;VIP[$i]&#125;</span> netmask 255.255.255.255 up</div><div class="line"> route add -host <span class="variable">$&#123;VIP[$i]&#125;</span> dev lo</div><div class="line"> <span class="keyword">done</span></div><div class="line"><span class="built_in">echo</span> <span class="string">"1"</span>&gt;/proc/sys/net/ipv4/conf/lo/arp_ignore</div><div class="line"><span class="built_in">echo</span> <span class="string">"2"</span>&gt;/proc/sys/net/ipv4/conf/lo/arp_announce</div><div class="line"><span class="built_in">echo</span> <span class="string">"1"</span>&gt;/proc/sys/net/ipv4/conf/all/arp_announce</div><div class="line"><span class="built_in">echo</span> <span class="string">"2"</span>&gt;/proc/sys/net/ipv4/conf/all/arp_announce</div><div class="line">&#125;</div><div class="line"><span class="keyword">function</span> <span class="function"><span class="title">stop</span></span>()&#123;</div><div class="line"> <span class="keyword">for</span> ((i=0;i&lt;<span class="variable">$&#123;#VIP[*]&#125;</span>;i++))</div><div class="line"> <span class="keyword">do</span></div><div class="line"> <span class="built_in">echo</span> <span class="variable">$&#123;i&#125;</span> <span class="variable">$&#123;VIP[$i]&#125;</span></div><div class="line"> ifconfig lo:<span class="variable">$&#123;i&#125;</span> <span class="variable">$&#123;VIP[$i]&#125;</span> netmask 255.255.255.255 up</div><div class="line"> route del -host <span class="variable">$&#123;VIP[$i]&#125;</span> dev lo:<span class="variable">$&#123;i&#125;</span></div><div class="line"> <span class="keyword">done</span></div><div class="line">&#125;</div><div class="line"><span class="keyword">case</span> <span class="string">"<span class="variable">$1</span>"</span> <span class="keyword">in</span></div><div class="line"> start)</div><div class="line"> start</div><div class="line"> <span class="built_in">exit</span></div><div class="line"> ;;</div><div class="line"> stop)</div><div class="line"> stop</div><div class="line"> <span class="built_in">exit</span></div><div class="line"> ;;</div><div class="line"> *)</div><div class="line"> <span class="built_in">echo</span> <span class="string">"You must use <span class="variable">$0</span>:stop|start"</span></div><div class="line"> ;;</div><div class="line"><span class="keyword">esac</span></div></pre></td></tr></table></figure>
<h3 id="测试实验结果"><a href="#测试实验结果" class="headerlink" title="测试实验结果"></a>测试实验结果</h3><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;如果测试部成功可以按照三角的排查原理来进行排查，显示client到RS端是否能通讯，LB到RS能否通讯，client到LB是否能通讯，client到VIP是否能够通讯。并且查看LVS的运行状态。一定要确保keepalived.conf这个配置文件是正确的。</p>

	

	
		<span class="different-posts"><a href="/2017/10/12/LVS/12. LVS原理详解及部署之五：LVS+keepalived实现负载均衡&高可用/" onclick="window.history.go(-1); return false;">⬅️ Go back </a></span>

	

</article>

	</main>

	<footer class="footer">
	<div class="footer-content">
		
	      <div class="footer__element">
	<p>Hi there, <br />welcome to my Blog glad you found it. Have a look around, will you?</p>
</div>

	    
	      <div class="footer__element">
	<h5>Check out</h5>
	<ul class="footer-links">
		<li class="footer-links__link"><a href="/archives">Archive</a></li>
		
		  <li class="footer-links__link"><a href="/atom.xml">RSS</a></li>
	    
		<li class="footer-links__link"><a href="/about">about page</a></li>
		<li class="footer-links__link"><a href="/tags">Tags</a></li>
		<li class="footer-links__link"><a href="/categories">Categories</a></li>
	</ul>
</div>

	    

		<div class="footer-credit">
			<span>© 2017 失落的乐章 | Powered by <a href="https://hexo.io/">Hexo</a> | Theme <a href="https://github.com/HoverBaum/meilidu-hexo">MeiliDu</a></span>
		</div>

	</div>


</footer>



</body>

</html>
